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ABSTRACT 



At the University of Delaware student -teacher communication 
within the classroom was enhanced through the implementation of a versatile, 
yet cost efficient, application of computer technology. A single 
microcomputer at a teacher's station controls a network of student 
keypad/display stations to provide individual channels of continuous 
communication from each student to the teacher. The innovation has 
capabilities far exceeding earlier A-to-E choice-tallying devices. This paper 
evaluates the technology at the postsecondary level, in the context of an 
introductory astronomy course. Uses for the technology include the 
assessment, during lectures, of diverse student attributes such as 
background, attitudes, misconceptions, specific preparation, and 
understanding of what has been presented. The most successful use was 
interactive lectures guiding students through multi-step numerical problems. 
The technology allows for more individual communication between the teacher 
and each student, showing the students what the teacher is expecting them to 
get from the lecture, and showing the teacher if the students are getting it. 
Appendices include a photograph of the keypads and tables of student 
evaluation results . (SWC) 
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Project Summary 



Computer Assisted Communication within the Classroom: 
Interactive Lecturing 
(FIPSE) PR/AWARD NUMBER P116B91706-91 



Student-teacher communication within the classroom is being 
enhanced by a versatile, yet cost efficient, application of 
computer technology. A single microcomputer at a teacher's 
station controls a network of student keypad/display stations to 
provide individual channels of continuous communication from each 
student to the teacher. This innovation, with capabilities far 
exceeding earlier A-to-E choice-tallying devices, is evaluated at 
the postsecondary level in the context of introductory astronomy 
courses. Uses include the assessment, during lectures, of diverse 
student attributes: background, attitudes, misconceptions, 
specific preparation, understanding of what has just been 
presented, etc. Most successful are interactive lectures guiding 
students through multi-step numerical problems. 
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Executive Summary 



Computer Assisted Communication within the Classroom: 
Interactive Lecturing 



Grantee Organization: University of Delaware, Newark, Delaware 19716 

Project Director: Richard B. Herr, (302) 831-2673, herr0brahms.udel.edu 



A. ' Project Overview 

In preparing a lecture, the professor programs into a laptop computer a 
selection of questions to use throughout the class. These include questions 
on the assigned material, questions to determine background knowledge, survey 
and evaluation questions, and, particularly, questions leading the class in 
steps through the lecture topic. 

When the teacher selects a question, each student responds through a 21-key 
(basically numeric) keypad and the response is judged by the criteria written 
into the computer program by the professor. The numerical responses of the 
students can be quite general; o.g., -2.746 x 10’*, as the answer to a 
quantitative scientific question. The professor's program sends a reply back 
to the display screen on each student's keypad. These replies are 
individualized to the judgement criteria, confirming that the student's answer 
was correct, or identifying a likely error, or giving a hint to try again, or 
giving the correct solution and answer. 

B. Purpose 

A variety of purposes are served by the system. It keeps students involved 
and can lead them through easy steps (assuring that each student has the 
correct intermediate answers) to the solution of multistep problems, more 
realistic to science than simple "formula" problems. Primarily, it is an 
avenue of more individual communication between the teacher and each student. 
It shows the students what the teacher is expecting them to get from the 
lecture and it shows the teacher if the students are getting it. 

C. Background and Origins 

After several years of writing stand-alone computer-assisted-instruction 
programs, I came to appreciate the enormous difficulty of creating a computer 
program that allowed for every possible student response, provided appropriate 
help when needed and never left the student frustrated by misunderstood 
instructions. If a teacher could be present while the student worked, these 
problems would be avoided. 

About the same time, 1980 to 1981, I was becoming increasingly disheartened by 
the difficulty of lecturing to the diverse group of students who take 
Introductory Astronomy. I would lecture without fully appreciating how many 
were unprepared, what deficiencies some had in their math and science 
backgrounds and what misconceptions some might be holding. 
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If only there were a way to get from every student in the class the kind of 
feedback that is elicited by an instructional computer program. The teacher 
would know where to put emphasis and be able to talk more effectively to the 
class. Also, all the student responses could be stored and used to give a 
more accurate evaluation of student performance than can be obtained from the 
few, albeit highly stressful, examination days. Students would continually 
know where they stand, rather than be surprised at the time of an exam by the 
professor's expectations. 

A student response system seemed a possible solution; but, in 1981 these were 
mostly multiple-choice tallying devices and did not allow the general 
numerical entry or the important two-way communication. A proposal was 
written to try to build more general keypads out of hand calculators but this 
was not funded. 



D. Project Description 

By 1989, more advanced response-system hardware had become available at an 
affordable price. The ALS keypads marketed by Reactive Systems were purchased 
through the FIPSE grant. These have five "function" keys (FI to F5) as well 
as 0-9 numeric keys, decimal, minus sign, etc. Their liquid-crystal display 
screens show the student what s/he has keyed in to send and they allow for 48 
characters of alphanumeric reply from the teacher's computer. However, 
software to take advantage of the keypad capabilities was not provided and had 
to be written under the FIPSE grant. 

More than 200 students have now taken astronomy at the University of Delaware 
using this system in some developmental form. Continual software improvement 
allowed more and more of the desired features to be implemented. The F5 
function key is set to display an E and to indicate the exponent of 10 for 
scientific notation; variable credit may be assigned (when desired) to 
different student answers; output files may be edited and scores imported to 
any spreadsheet. 

In the first semester both students and professor criticized the overhead of 
wasted class time required for the daisy-chain hookup of the student keypads 
and the connection of the professor's desktop computer wheeled into the 
classroom on a cart. Such wasted time has now been reduced to an 
unobjectionable minimum by installing permanent room wiring with outlets at 
the student desks and by the professor using a battery powered laptop computer 
requiring only one easy connection at the lecture table. 

E. Project Results 

As might be expected for any departure from familiar teaching methods, some 
student reacted quite positively, some negatively but most accepted it 
dispassionately. On average, student response was more positive than negative 
when they rated the system on how helpful it was to their learning astronomy. 
Tost scoros for quGstions that had also bGGn askod to classGS prior to usg of 
the response system were disappointingly similar to those of the earlier 
classes. Also discouraging was the fact that less subject material could be 
covered than when the class periods were filled with lecture that was 
virtually uninterrupted by student interactions. This was expected and 



accepted as the price for covering selected material in greater depth. No 
control group was possible for this new coverage; but, although students gave 
their highest endorsement to these interactive-lecture methods for multi-step 
problems, assessment of the knowledge they acquired made it dubious whether 
the work required to create such lectures could be justified. 

The system will continue to be used and improved. A more thorough statistical 
evaluation of the enormous body of accumulated data is planned for 
publication. Because (as described elsewhere) of the delays caused by 
programming difficulties and the disallowal of a no-cost extension as a result 
of communication problems within FIPSE, a more complete analysis is not yet 
possible. 



F. Summary and Conclusions 

Questioning students to keep them involved in the reasoning steps to a deeper 
understanding of a topic in not a new pedagogical technique since it dates at 
least to the time of Socrates. Obviously, it works best when only a few 
students are involved. The oration or lecture is equally old and a 
marvelously efficient means of informing a large group of people. This 
project has shown that today's computer technology can combine elements of 
both methods. 

That combination was found to require an extraordinary amount of work on the 
part of the teacher to prepare presentations for classes. Questions 
representing the best instructional use of the system have to be created and 
completely programmed into an error-free computer file. The most time 
consuming part of this programming is devising the proper judgement criteria 
in anticipation of student answers and then phrasing (in three short lines of 
16 characters each) appropriate replies to the correct and incorrect answers. 
In addition, it typically required one to two hours of extra time after each 
class for follow up work. 

We are not considering here the one-time hardware and software development 
time for the overall project. Nonetheless, it was found that most of the 
above-considered time surrounding each class was not greatly reduced in the 
second or third years even though many of the same questions were reused. By 
the Fall, 1993 semester, just concluded, a more relaxed usage of the keypads 
was found to be most functional. Deciding not to use a question was much more 
acceptable to me if I had composed it in a previous year rather than having 
just spent an hour before class laboring over it. 

In balance, the system does contribute to better teaching. Student 
preparation for class, attentiveness in class and understanding of multistep 
problems all seem improved but not sufficiently to justify the work required 
of the teacher. Attendance and classroom discussion are definitely improved, 
but, again, the bottom line of demonstrably better student performance is 
disappointing. It will continue to be used and improved in my astronomy 
classes but with less emphasis than when I felt compelled to make each class 
an exhaustive trial of the methodology. 

G. Appendices 

Photograph of keypads and tables of some student evaluations. 
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Final Report 



Computer Assisted Communication within the Classroom: 

Interactive Lecturing 

A. Project Overview 

The roots of this project date back to 1981 when I realized that computer 
technology might provide a solution to some problems that plagued my teaching. 
My colleagues confirmed these to be common problems, perhaps inherent in a 
system of mass education. Despite my best efforts, I found that student often 
came to my classes poorly prepared; their minds tended to wander during class; 
they missed important points. That meant that they couldn't understand a 
later concept and their difficulty mushroomed! I would lecture on without 
really knowing their weaknesses in background, in preparation and in missed 
concepts. 

A test would loom ahead! Panic cramming to retain as much as possible through 
just one short, but CRITICAL hour. Only then did a clearer picture emerge of 
how much less I had taught than I thought I had taught. I and my students 
would be mutually discouraged as we resumed the pattern for the next test. 

My dream was of a system by which, at all times during the class, each student 
would have a private channel of communication with the teacher. By virtue of 
a FIPSE grant, this dream became the reality of a classroom wired for student 
response/display pads connected to a central microcomputer operated during 
class by the teacher. 

Although the experiment of how best to use this facility is still in progress, 
supported by University of Delaware matching funds through May, 1994, 
sufficient experience has been gained through the Fall, 1993 semester to write 
a "final" FIPSE report now that that funding has ended. 

Approximately 200 student have now (December, 1993) completed courses taught 
using this system. Of these, 141 took "Introduction to Astronomy I" (PHYS 

133) offered in the fall of 1990, 1991, 1992 & 1993. This is the first 
semester of our most quantitative introductory astronomy course at the 
University of Delaware and is offered in the usual format of three lecture 
sessions and one laboratory session each week. In the spring semesters (PHYS 

134) of 1991, 1992 & 1993, 59 students used the system. Additionally, 48 
students took PHYS 133 and 18 students took PHYS 134 in the academic year 
1989-1990 when the system was only partly in place. Since September, 1990, 
the basic system has not changed dramatically although there have been 
continual improvements to both the hardware and the software that facilitate 
its use. This report, therefore, concentrates on the pedagogical results in 
this later period. 

At this time, my conclusion is that the system is of value and will continue 
to be used and improved but that the bottom line of improved student 
understanding of astronomy was not sufficient to justify the extraordinary 
effort required by the teacher to implement the method. 
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B. Purpose 



Science, like mathematics and language, is learned progressively. That is, 
many concepts cannot be appreciated without first understanding preliminary 
material. A student who falters on a lower step is unable to climb to the 
levels expected for postsecondary education. Although this progression is 
widely appreciated in math and language instruction, and is appropriately 
addressed in elementary and secondary schooling, this does not seem to me to 
be the case as often in science. I find students thinking that they can 
understand, or that they do understand, black holes without first attaining 
some basic knowledge about the nature of mass, force, gravity, electromagnetic 
radiation, etc. 

This trend may continue in some college astronomy courses where the emphasis 
is so heavy on description, rather than on science, that the topic seems 
indistinguishable from mythology: the earth in space is described and 
understood on no more scientific a basis than some story of our living on the 
back of a giant turtle in an endless sea. 

When quantitative material is introduced in a course, student have been 
conditioned to expect it to be in the form of "formulas" which they must 
memorize. The "word problem" is expected to contain a "ham-and-eggs" word 
association reminding them of the formula to use and should reinforce this by 
containing the values for all of the quantities on the right-hand side of the 
formula (and for nothing else). Even the semantics is antithetical to science 
- the idea that these are application "formulas" rather than relationships 
among physical quantities in the universe. 

My purpose in the interactive- lecture project has been to dispel these 
misrepresentations of my field and to guide students to a more valid 
appreciation of what actually constitutes science. After some twenty years of 
attempting to do this using (mostly) traditional methods of college teaching, 

I was convinced that I was bucking a system that is comfortably non-demanding 
for both students and teachers. 

The picture presented in the preceding paragraphs is grim - the worst case 
situation. However, addressing a solution is not made easier by the fact that 
not all students have had so crippling a preparation for higher education in 
science. What is the best compromise between attending to the remedial needs 
of some students without stifling those prepared to gain the most from the 
course? Unlimited individual attention is an unattainable ideal. Might 
technology help achieve the desired individual communication while preserving 
the wonderful efficiency of lecture presentations for mass education? 

Many educational buzzwords were apropos to my proposed project: communication, 
motivation, active learning, nondiscriminatory involvement of the entire 
class. All of these goals were, in fact, enhanced by the program. The core 
structure is (1) a question is asked by the teacher, usually via viewgraph, 

(2) each student enters a response on his/her keypad, (3) the responses are 
recorded and judged by a previously prepared program in the teachers computer, 
(4) each student receives back a reply based on the response that s/he 
entered. 
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C. Background and Origins 



The conceptual origins of the project have been described above. In 1981 I 
knew of no satisfactory keypads marketed to do what I wanted. That is to 
allow students to enter any number; e.g., -4.621 x 10’®, and have it be judged 
by the instructor's computer. I considered building buffer chips into 
inexpensive hand calculators so that the displays could be read by a central 
microcomputer, but that 1981 project was not funded. By 1989 only one vendor 
(Reactive Systems) had emerged to provide what seemed a viable off-the-shelf 
unit. They made several models, the most versatile being the Advanced 
Learning System (ALS). An ALS keypad consists of an LCD screen, with four 
lines of 16 characters each, and 21 keys including 0-9 numeric keys, a help 
key and five programmable function keys. The pads are connected by telephone 
wire and standard modular (RJll) telephone plugs to a multiplexing unit that 
interfaces to an IBM-compatible microcomputer through the RS-233 serial port. 

In comparison to the components contained in a $15 hand calculator, the $210 
price per keypad/display unit could only be justified by the limited market. 

We bought a multiplexer and 64 keypads which arrived in September, 1989. 

Unfortunately, the software supplied by the vendor limited the keypads to 
nothing more than a simple multiple-choice tallying system. The inability to 
use the function keys, the help key, minus signs, decimal points, and even to 
enter multidigit numbers made the $210 price tag outrageous. Our programmer 
who was to customize the vendor's response acquisition program was overwhelmed 
and had to start from scratch. Reactive Systems seemed unable to fully solve 
the problem, although by January 1990 they had supplied us with computer 
routines that we could incorporate to allow sending more general numbers. The 
help and function keys were still not recognized, even locally within the 
keypads. A second programmer worked on this problem attempting to write the 
code to actually reprogram the integrated circuits in the keypads. But it 
wasn't until September, 1990 (a full year after the pads were purchased) that 
Reactive Systems was able to supply us with an upgraded EPROM and gave us 
permission to copy its microcode into our keypads. 

During the first semester of trials (Fall, 1989), the students daisy-chained 
their keypads, plugging one into the next to complete a circuit from the 
computer interface. This works fine for a small class or for a system that 
has to be portable; we used it successfully for a November 2, 1991 
presentation at a FIPSE Project Directors' Meeting. But, even in a class of 
only 30, the hookup took an annoying amount of class time and introduced 
frustrations when someone didn't properly make the connection. Much effort, 
went into creating a permanent wiring scheme for our lecture hall. By the 
spring of 1990, surface wiring had been installed with telephone style outlets 
at every other seat providing 100 receptacles for keypads. In March, 1991 we 
purchased 33 additional keypads and a second interface box. Although it seems 
unlikely that the ALS system will ever be able to address more than 64 keypads 
(contrary to the initial expectations from Reactive Systems) the extra pads 
and interface have proved most desirable as a hardware backup and to allow 
program development and class preparation on an out-of-classroom system. 

Next to the delays created by the insufficiency of the software supplied by 
the vendor to support their keypads, the most serious obstacle to progress was 
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the loss of our programmer in May, 1991. Attempts to find a satisfactory 
replacement during the following academic year were unsuccessful. Although a 
couple of student programmers attempted work on the response acquisition 
program, no usable additions were produced. One promising programmer did 
complete an auxiliary utility program to make it easy for teachers to create 
the question/response files needed for each lecture. Unfortunately he left 
the university before he could contribute to the response acquisition program. 

To assure that the remaining features desired in the program would be 
completed, a written request for a one-year, no-cost extension of the project 
was submitted to the FIPSE Progr-am Officer on June 4, 1992. This was 
confirmed in a FAX received June 12, 1992 requesting budget details which 1 
FAX'ed back on June 17, 1992. Obviously, a more experienced (and more costly) 
programmer was needed, so approval from FIPSE was obtained, dated December 4, 
1992, to merge the remaining money to pay a senior programmer to complete as 
many of the improvements as the remaining money would allow. Unhappily, the 
request for no-cost extension was disapproved on the basis of it not having 
been received prior to July 31, 1992. My March 22, 1993 petition for 
reconsideration pointed out that, although my extension request had not been 
transmitted to a Grants Officer until after the deadline, 1 had submitted it 
to my Program Officer in a timely manner. Nonetheless, the extension was not 
approved. An expert programmer was obtained in the fall of 1993 and is being 
paid with university funds remaining in a matching account. 



D. Project Description 



CONCEPT: Throughout a lecture class, students are asked questions to which 

they respond on individual keypads and receive the teacher's preprogrammed 
replies from a central computer which stores all the data for analysis. 

HARDWARE: 21-key, 4xl6-character display, numeric keypads from Reactive 

Systems; IBM-compatible laptop computer; Reactive Systems's interface; and 
local room wiring. 

SOFTWARE: Locally written Response Acquisition Program (RAP), Microsoft 

Professional BASIC Compiler; spreadsheet (SuperCalc) for maintaining scores. 

PROCEDURE: The teacher brings to the classroom a laptop computer containing 

RAP (see above), a class-roster-initialization file, and a specific question & 
response- judgment file for the day's class (the .DAY file). This computer is 
plugged into an interface box which is permanently located in the classroom. 

As students enter the room they select from a cart of boxes their individually 
numbered keypads, sit where they like and plug their keypads into outlet boxes 
behind their seats. 

The interactive lecture session begins when 
the teachers enters "rap" on the laptop 
computer. RAP accesses the class roster 
initialization file (INIT.MSG) which 
attaches student names to pad numbers and 
which may also contain initial 
individualized messages to be sent to each 
student's display screen as in the figure. 

Students receive their personalized 
messages, which also provide confirmation 
that they have correctly picked up the 
keypads that were assigned to them. The 
messages were written into the INIT.MSG 
file by the teacher prior to class using 
any ASCII text editor. 

A typical class may start with the professor asking if anyone has questions on 
the assigned reading for the day. Students respond through the keypad by 
sending the page number of the material that they don't understand. The 
students' names and the page numbers appear on the professor's computer screen 
and the professor responds to the questions. The list of student names and 
the pages that they questioned are saved in the output file for later 
reference as to who was participating in class and what readings from the text 
gave the most trouble. 

When there are no more questions from the students, the professor may ask 
several questions selected as important from the assigned material or from 
review material. The most practical mechanism for presenting questions was 
found to be the ubiquitous overhead, "viewgraph," projector. Each student 
keys in a response which, depending upon the question, may be a numerical 



-I Eliot 


Thomas 


: Hello Tom, 
See me about 
; math review. 




: -2 Prufrock. 
: Hello Al, 
Keep up the 
: good work 1 


Alfred 


'■ .. .etc. 





Sample from an INIT.MSG file. 



1 

X JL 
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answer, a multiple-choice selection, a yes/no or true/false. The responses 
are judged by the criteria that were entered in the .DAY file by the teacher 
prior to class. RAP as presently written permits up to eight judgements to 
distinguish different answers. 



Figure 2 shows one question as coded 
in a .DAY file by the teacher using 
a text editor or using our 
aforementioned user-friendly utility 
for creating .DAY files. This 
example shows full use of the 
options to (1) assign credit points, 
differing for different responses; 

(2) send appropriate replies back to 
the students; (3) reflect back the 
number that the student sent; and 
(4) permit students to try again, in 
which case their later answer is the 
one that counts for the credit. 

The beginning of a new question is 
coded as - in column 1 (the leftmost 
column of a line). Following that 
line is a full statement of the 
question terminating when a line is 
encountered beginning with !. This 
! is the code for a judgement line 
on which must appear a numerical 
expression; e.g., y=7.2E-6, giving 
the judgement criterion. In this 
case, if the student had sent the 
number 7.2E-6 (or .0000072) he or 
she would receive back whatever 
lines (up to 3) are in the .DAY file 
between that judgment line and the 
next ! or - in a column 1. In the 
reply to the student, <answer> is 
the code for sending back to the 
student the value that the student 
sent. 

Optional characters on the judgement 
line are [] which enclose a point 
score that will be given for that 
answer and * which means that the 
student may try again using the clue 
on his/her screen. The example in 
Fig. 2 uses this device in a 
lighthearted way to direct all 
students to the correct answer. 



-Latitude of Washington? 

What is our latitude (in degrees) 
here in Washington, D.C.? 

i38<=x<=39 [2] 

Congratulations ! 

U know where U R 
Approx 38.9 deg. 
i*36.5<=x<38 [1] 

Close! You're in, 

VA; go a bit to 
north. Try again 
!*39<x<=42.4 [1] 

Close! . Come 
south a bit on 
195. Try again. 

!*70<=x<-80 to] 

That looks like 
longitude. Try 
again for laf. 

!*x<=0 [0] 

TryAgain. U sent 
<answer> 

O.C. Idt. is >0. 

!*x<25.8 [0] 

You are south of 
Miami. Go north 
and try again. 

!*x>42.4 [01 
You are north of 
Boston. Co south . . 
and try again. . 

I*25.8<=x<36.5 [0] 

You are in deep 
Dixie. Travel 
north*, try again 



Fig. 2. Sample question in .DAY file. 
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While running RAP, the teacher selects the option of asking questions from a 
.DAY file, then sees a list of the .DAY files in the computer directory, 
selects the desired .DAY file (usually named with the date; e;g., DEC7.DAY) 
and is presented with a list of that file's questions. Questions are listed 
in the order in which they occur in the .DAY file and are identified by the 
text that appears (see Fig. 2) after the - which is the code for the start of 
a question. When the cursor is placed on a question identifier in the list, 
the more complete question text (in the lines following the - line in the .DAY 
file) appear in a box on the instructor's screen. If <Enter> is pressed, that 
question has been selected, polling of the keypads begins and response 
judgements are those for the selected question. 

As students send their responses, their names and responses appear on the 
instructor's screen. Students do not see this list. When the teacher sees 
that all, or most, of the students have responded, or that a time limit has 
been reached he or she ends the polling and is presented with a bar graph 
showing the number of responses in each of the judgment categories. All of 
this information is saved in an answer (.ANS) file: the question statement, 
each student's response and name in the order received, and the bar graph of 
results. A separate output file contains just the student scores (not all 
questions are scored) in a matrix suitable for importing to a spreadsheet 
where analysis can be performed. 

Spontaneous questions (not in the .DAY file) may also be asked but student 
answers for these cannot be judged or scored. In this case a default 
acknowledgement; e.g., "Your answer <answer> was received," is returned to the 
student keypads. 

Class attendance, of course, is automatically recorded with latecomers, who 
plugged into the system after the roster initialization, noted in the name 
list. The lecturer can see this attendance list at any time except when 
active polling is occurring. 

Message files other than INIT.MSG can also be sent to the class during the 
session. An important use of this is to confidentially send to each pad the 
student's total present score and grade for the course. Message files for 
this purpose are easily prepared from the grading spreadsheet (which weights 
the student's classroom performance with test and lab scores) using macros. 
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E. Project Results 



Within the procedural framework described above, diverse teaching options are 
possible. In this section I will describe some of the things that I have 
tried and how well they have worked. The fact that new ways to use the system 
are continually presenting themselves, leads me to believe that there are many 
other (very possibly, better) uses of the system than have yet occurred to me. 
The potential seems great and likely to become even greater as the technology 
to exchange more complex communications in the classroom becomes more 
affordable. 

Because of the credit points assigned to some of the daily questions, class 
attendance is higher than in classes where classroom performance is not so 
routinely evaluated and used as a component of the final course grade. The 
format of having, essentially, a daily quiz encourages students to keep up and 
to be prepared for class. This is certainly not a new discovery, but the 
keypad system facilitates delivery and provides immediate feedback of quiz 
results to the teacher before beginning a lecture. 

Allowing virtually unlimited questioning by students before such quizzes, has 
engendered more student discussion than I have ever had before in an astronomy 
class. I have sometimes spent the entire class period responding to good, 
legitimate questions (as opposed to those introduced to delay quiz questions). 
Quiz-delaying questions are usually transparent and readily dealt with. 
Students know that I will completely answer a question even if it is one that 
I plan to ask them. In fact, I encourage them to try to outguess me, to get 
on the same wavelength as to what is most important from the reading 
assignments. It may even be argued that learning to read a science book is a 
more valuable accomplishment than learning the specific factual material from 
the course. 

A major motivation for this project was to experiment with a more Socratic 
lecturing style where students are led to an understanding of the subject 
through a sequence of questions. Prior to lecturing on a topic that requires 
use of the trigonometric sine, the teacher might ask a non-credit question 
ascertaining that class members know the mathematical definition and can solve 
for a numerical answer. Depending on the responses received, the teacher 
might spend more (or less) time reviewing the prerequisite math. Students 
missing the question might be directed, through a keypad message, to a math 
review exercise set up on an out-of-class computer. 

Questions to determine ongoing understanding of the lecture may be sprinkled 
throughout the lecture. The goal is severalfold: to keep the teacher 
appraised of student progress so that the lecture remains appropriately 
directed to the audience, to keep the students actively engaged in the 
learning progression and to give them stimulated opportunities to ask 
questions. Finally, questions toward the end of the session may carry credit 
as a test of comprehension and ability to use the concept. 

Often a question builds toward a later question to establish first that each 
student understands the vocabulary or knows a prerequisite piece of factual 
information. For example. Figure 2 contains a question that might be used to 



lead toward a question testing whether students realize that the altitude of 
the north celestial pole (essentially the altitude of the North Star, Polaris) 
is a measure of the observer's latitude. The Figure-2 question directs every 
class member to the correct value for the latitude of Washington, D.C. The 
next question could establish for everyone the definition of "altitude" by 
asking, "What is the angular distance above the horizon of a star whose 
altitude is given as 23 degrees? Give your answer in degrees." Then, the 
focus question, "What is the altitude of Polaris as seen from Washington, 
D.C.?" The idea may be reinforced by a follow-up question, "Norfolk, VA is 2° 
of latitude south of Washington. What is the altitude of the north celestial 
pole at Norfolk?" 

Because a live instructor is present when such questions are being asked, oral 
discussion and clarification is always possible. Thus, although preparation 
of a bug-free .DAY program is very time consuming, it is not as overwhelming 
as preparing a self-paced, stand-alone tutorial. Feedback from the students 
is immediate for each question. 

Ideally, students (1) study the assignment before class, knowing that one 
aspect of the response system will be a quiz on assigned material; (2) use the 
initial questioning period in class to clarify material that they didn't 
understand from the reading; (3) follow the lecture using the interactive 
system to check that they are getting the main ideas and to accumulate daily 
credit that will make their final grade less dependent on a few tests. 

We have called this interactive or participatory lecturing; it combines 
feedback elements of tutorial CAI with the explanatory discourse of the in- 
person teacher. 



F. Summary and Conclusions 

Ongoing evaluation was an inherent part of the project. In the Spring, 1992 
semester, midway through the project, oral interviews were conducted, both 
individually with each student (by the professor) and with the class as a 
whole (by a professional from our Center for Teaching Effectiveness, with the 
professor absent). Student evaluations also included a formal, anonymous 
year-end evaluation (every semester), mid-semester written critiques by the 
students (most semesters) and evaluations conducted throughout every semester 
using the classroom keypads to obtain student opinions. A brief "PreTest" was 
given on the first class day of ^ach course as a measure of student 
backgrounds (along with the demographic information available from the 
University Records Office). Every classroom response that every student made 
on the keypads during the entire course of the semester has been retained in 
computer retrievable form along with their answers to written test questions. 
Written tests were prepared to be comparable to those given in previous years 
as a partial control on current achievement. All tests contained some 
questions identical to those given to a pre-keypad class and several old tests 
were found that could be given almost entirely as photocopies of the original 
exams. Twenty years of old exams were available, excluding the last three 
years for which copies are placed on reserve for student study. Item analysis 
has been performed on both the present exams and on the control exams. 

Interactive-lecture methods were continually modified based on this feedback 
from the students. Creating the kind of sessions that were instructionally 
most helpful (as judged by both students and the teacher) was extraordinarily 
time consuming. If testing the best use of the computer-assisted response 
system was to be the focus of the project, then the irreducible first priority 
was an effort to attain optimum use of the system. I /eel satisfied that this 
was achiGVGd to thG bGst of my abilitiGS within thG limits imposGd by timG and 
money. However, this meant delaying the preparation of reports for 
publication or bringing together for comprehensive analysis the extensive 
evaluation data that was being accumulated. A graduate student specializing 
in educational evaluation was to be paid to analyze the massive body of data. 

A procedural plan was created; but, because of the disallowance of the no-cost 
extension, this activity has been deferred until an alternative can be 
devised. 

Nonetheless, I will risk setting forth here some tentative conclusions which 
derive from the diverse appraisals used throughout the project for polishing 
my teaching methods. It remains to be seen if these conclusions are supported 
by a thorough analysis of the data. This is written as a frank and informal 
summary of my present thinking. 

I judge the project a mixture of success and failure. In broad terms the 
success lay in the enhanced ability to communicate repeatedly with each class 
member during the lecture session. Because student responses often counted 
toward the course grade, attendance and attentiveness were exceptionally good. 
But, the goal that was most gratifying to reach was the ability to hold the 
class together in progressing through multistep problems. Our educational 
system has been criticized, most rightfully in my opinion, for not teaching 
realistic problem solving that involves identifying and solving intermediate 
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problems to arrive at the desired answer. Addressing this deficiency was an 
ultimate ideal of the project. The best of interactive lecturing breaks the 
larger problem into small decision and calculation steps, each of which can be 
discussed with the class, and a solution path established. The students 
regroup at each way station before beginning the next leg of the journey. ^ 
Multistep problems, however appropriate to a correct representation of science 
at the postsecondary level, tend to be avoided in conventional teaching 
because of the ease with which nonscience majors get lost along the way. 

The failure, again in broad terms, lay in the inordinate amount of time that 
it takes to program such a multistep-problem sequence. After a subject is 
chosen as suitable, a storyboard must be created for the way that the steps 
will progress. Each step must be crafted with the care of an exam question to 
minimize ambiguities and to stimulate thought. But the bulk of the time is 
usually in formulating answer judgements and composing appropriate replies 
individualized to these answers (and constrained to three lines of 16 
characters each) to send to the student screens. If some responses are to be 
scored for credit, the number of points assigned to the responses must be 
coded into the day program and included in the message to be sent back to the 
student. The presentation materials must then be prepared including overhead 
transparencies of the questions and of supplementary information. Because the 
computer program for the interactive lecture could be completely unusable 
because of a single programming or typographical error, it must be tested with 
a small set of keypads. At this point I frequently find improvements to make 
in the wording of the replies as well as in the logic of the judgements. The 
total time, including presentation, post-editing, record keeping, and a 
minimal evaluation can amount to several days work for a single class meeting 
when such an extensive interactive lecture is offered. 

In spite of the time required, if the effort were rewarded with outstandingly 
improved student achievement, the project could be considered highly 
successful. We all know that it takes many, many hours to produce a single 
hour of tutorial computer-assisted instruction. But, my feeling is that 
student achievement was only marginally improved and even that may be hard to 
document with convincing evidence. In short, the results do not seem to 
justify the effort unless the interactive- lecture programs that were prepared 
are used repeatedly for a larger number of classes. 

Most distressing was the constant reminder that some students could not be 
reached whatever I did. It may not be politically correct to verbalize this, 
but it is my reluctant conclusion that some attenders of our postsecondary 
institutions just cannot be motivated to an actual enjoyment of the 
intellectual activity demanded for higher education in science. In devising 
interactive lectures for multistep-problem solutions, my objective is to 
create intermediate questions which, although thought provoking, have fairly 
obvious, or, at least, straightforward answers. The idea is that most 
students are answering correctly these easy steps and are feeling good about 
their progress as we merrily progress to the shining solution at the edge or 
the forest. Of course, there is a limit to how simplistically one can break 
down a problem into trivial ministeps before one begins to cheat those 
students who are already motivated and capable of greater challenges. If 
interactive lecturing holds back these students, it should be buried quickly 
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before it can do further damage. Elective introductory courses attract a 
clientele with an extreme range of backgrounds and abilities. My dilemma in 
finding the best compromise for teaching so diverse a group was that, no 
matter how trivial I made the ministeps, there were some students who would 
consistently miss them. After nearly 30 years of teaching, this should not 
come as a surprise to me, but a negative aspect of interactive lecturing is 
the very fact that class results are immediately displayed for the teacher's 
evaluation. To knowingly leave behind some students or to risk boring the 
majority with continual repetition of the elemental ideas is a frustration the 
interactive lecturer must be prepared to face -- and to contain, because it is 
not productive to display such frustration very often before the class. 

Indeed, I came to suspect an element of counterproductivity in my efforts to 
reach all class members. That is, the more it was apparent that I considered 
it my mission to teach them astronomy, the less the responsibility they felt 
they needed to exercise toward that goal. 

I was pleased to find that the students appreciated these sessions on 
multistep problems. It surprised me that even the best students considered 
them worthwhile, in spite of my trying to lead them into expressing negative 
criticism by pointing out (in the private interviews) that more material could 
be covered in class by traditional lecture methods. These student opinions 
were valued, but I would have preferred more concrete evidence that greater 
learning was being attained. It is difficult to assess the value of 
interactive lecturing in this area of multistep problems because I have no 
control group in which this objective was attempted by traditional methods. 

In a wider context, the project also has value because of its obvious 
extendibility to distant learning where some, or all, class members are 
connected via phone line or fiber optics from remote sites. Also, with the 
increasing application of multimedia techniques and computer technology to 
lecture presentations, the methods of interactive feedback are likely to 
become a common component of such classes. This FIPSE project has accumulated 
data concerning the strengths and weaknesses of interactive lecturing. 



G. Appendix 




Photograph of keypads with personal messages as seen 
by students at the beginning of class. 
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Table 1 



Number of Fall Semester Astronomy Students 
Responding to Evaluation Question in Two Years in Which an 
Evaluation was Requested Just Before the First Exam and Also at the End of the Semester 



"On a scale of 1 to 9, how helpful do 
you feel our system of interacting 
through the keypads is to your learning 
astronomy?" 


"Was the ’Interactive Lecturing" 
response-pad system worthwhile?" 


Just before Exam 
1 (Oct. 4-7). 
Keypad question. 


1991 


1992 


Anonymous 
questionnaire at 
end of semester. 


1991 


1992 


(9) “ very, very hcipftil 


0 


1 


(8.5) “ Strongly Positive 


2 


12 


(8) 


2 


0 








(7) — very helpfiil 


7 


10 


(6.5)“ Positive 


11 


5 


(6) 


7 


16 








(5) “ neither more nor less helpfiil 


5 


4 


(5 . 0) “ Neutral 


7 


4 


than regular class methods 












(4) 


3 


4 


(3.5)= Negative 


5 


5 


(3) “ very unhelpfiil 


2 


4 








(2) 


0 


0 


( 1 . 5) = Strongly Negative 


3 


4 


( 1 ) ” very, very unhelpfiil 


1 


1 








(Average Value) 


(5.59) 


(5.60) 


(Average Value) 


(5.20) 


(5.93) 



By the end of the semester in 1992 we had won over a significant number of students to feeling 
strongly positive toward the keypad system; however, there continued to remain a group who 
were not happy with this change from methods with which they were familiar. 
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Table 2 



Numbers of Fall Semester Students 
Responding to Study-habits Questions 



Compare the time that you spend studying for this course 
in relation to other courses. 


(Value) 


1989 

Oct.4 


1990 
Oct. 19 


1991 

Oct.4 


1992 

Oct.7 


1993 
Dec. 10 


More for this (1.00) 


10 


15 


8 


8 


11 


Same as others (2.00) 


27 


12 


17 


20 


6 


Less for this (3.00) 


10 


6 


2 


13 


3 


(Average Value) = 


(2.00) 


(1.73) 


(1.78) 


(2.12) 


(1.60) 



If we did not use the response pads, 
I would probably spend 


(Value) 


1989 

Oct.4 


1990 
Oct. 19 


1991 

Oct.4 


1992 

Oct.7 


1993 
Dec. 10 


More time (1.00) 


- 


1 


5 


6 


1 


About the same (2.00) 


- 


23 


15 


25 


14 


Less time (3.00) 


- 


9 


7 


10 


5 


(Average Value) = 


- 


(2.24) 


(2.07) 


(2.10)^ 


(2.20) 
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Table 3 



Numbers of Fall Semester Students 
Responding to Attendance Questions 



Compare your class attendance for this course 
to that for other, similar, courses 


Attend (Value) 


1989 

Oct.4 


1990 
Oct. 19 


1991 

Oct.4 


1992 

Oct.7 


1993 
Dec. 10 


this more often (1.00) 


14 


9 


7 


11 


8 


about the same (2.00) 


33 


22 


20 


28 


11 


this less often (3.00) 


2 


0 


0 


1 


0 


(Average Value) = 


(1.76) 


(1.71) 


(1.74) 


(1.75) ^ 


(1.58) 



If we did not use the response pads, 
I would probably attend 


(Values) 


1989 

Oct.4 


1990 
Oct. 19 


1991 

Oct.4 


1992 

Oct.7 


1993 
Dec. 10 


More often (1.00) 


- 


0 


0 


1 


1 


About the same (2.00) 


- 


25 


20 


25 


14 


Less often (3.00) 


- 


6 


7 


12 


5 


(Average Value)- 


- 


(2.19) 


(2.26) 


(2.29) 


(2.20) 
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